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Abstract. We complete previous investigations on the dynamical stability of barotropic stars and collisionless 
stellar systems. A barotropic star that minimizes the energy functional at fixed mass is a nonlinearly dynamically 
stable stationary solution of the Euler-Poisson system. Formally, this minimization problem is similar to a con- 
dition of "canonical stability" in thermodynamics. A stellar system that maximizes an H-function at fixed mass 
and energy is a nonlinearly dynamically stable stationary solution of the Vlasov-Poisson system. Formally, this 
maximization problem is similar to a condition of "microcanonical stability" in thermodynamics. Using a ther- 
modynamical analogy, we provide a derivation and an interpretation of the nonlinear Antonov first law in terms 
of "ensembles inequivalence": a spherical stellar system with / = /(e) and /'(e) < is nonlinearly dynamically 
stable with respect to the Vlasov-Poisson system if the corresponding barotropic star with the same equilibrium 
density distribution is nonlinearly dynamically stable with respect to the Euler-Poisson system. This is similar to 
the fact that "canonical stability implies microcanonical stability" in thermodynamics. The converse is wrong in 
\q ' case of "ensembles inequivalence" which is generic for systems with long-range interactions like gravity. We show 

that criteria of nonlinear dynamical stability can be obtained very simply from purely graphical constructions by 
using the method of series of equilibria and the turning point argument of Poincare, as in thermodynamics. 
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^ ' 1. Introduction are often treated in parallel although these systems are 

physically different (Binney & Tremaine 1987). 



In this paper, we shall tackle these dynamical stabil- 
The dynamical stability of stars and galaxies is an impor- ity problems by a different approach, using optimization 
^ tant topic in astrophysics. A large number of results have principles. These optimization principles provide condi- 
' been obtained in the past and are collected in Binney & tions of nonlinear dynamical stability for barotropic stars 
Tremaine (1987). Most of these results are based on a and spherical galaxies. This is different from the ther- 
linearization of the Euler-Poisson and Vlasov-Poisson sys- modynamical stability of stellar systems (Padmanabhan 
terns around a stationary solution and on the study of 1990). In the thermodynamical context, we maximize the 
the evolution of the perturbation which is decomposed Boltzmann entropy Ss[f] = — /m m m^ r ^ v a ^ ^ xe< ^ 
in normal modes e <T '/i (T (x). In practice, one has to work mass M[f] and energy E[f] in order to obtain the most 
out an eigenvalue equation determining the complex pul- probable equilibrium state (/ ~ e~^ me ) of a iV-body 
sation a and the corresponding eigenfunction /i CT (x). This Hamiltonian system resulting from a "collisional" relax- 
approach provides a condition of linear dynamical stabil- ation for t 3> t re i ax , where t re i ax ~ (N/hiN)tr) is 
ity when R e (a) < for all er and a condition of instability the Chandrasekhar relaxation time (to is the dynami- 
otherwise. It has been found that a spherical stellar system cal time). In that case, the Boltzmann entropy is ob- 
with DF / = /(e) and /'(e) < 0, where e = v 2 /2 + ^(r) is tained from a combinatorial analysis (Ogorodnikov 1965). 
the individual energy, is linearly dynamically stable with On the other hand, the maximization of any ii-function 
respect to the Vlasov-Poisson system if the barotropic gas S[f] = — J C(f)drdv, where C is convex (i.e. C" > 0), at 
with the same equilibrium density distribution is linearly fixed mass and energy determines a nonlinearly dynami- 
dynamically stable with respect to the Euler-Poisson sys- cally stable stationary solution / = /(e) with /'(e) < 
tern. This forms the Antonov first law (1960) and explains of the Vlasov-Poisson system which describes collision- 
why the stability of stellar systems and gaseous systems less stellar systems for t <C t re \ ax . Formally, these max- 
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imization problems are mathematically similar (the sec- 
ond one involving a larger class of functionals than the 
Boltzmann functional) but their physical content and in- 
terpretation are completely different: the first one refers 
to thermodynamics (collisional regime) while the second 
refers to dynamics (collisionless regime). However, due 
to their mathematical similarity, we can develop a ther- 
modynamical analogy (Sec. I4.1fl to analyze the nonlinear 
dynamical stability problem. In this analogy, the crite- 
rion of nonlinear dynamical stability for spherical stellar 
systems with / = /(e) and /'(e) < described by the 
Vlasov-Poisson system (Sec. 12. 211 is similar to a criterion 
of thermodynamical stability in the microcanonical en- 
semble. We have also found that the criterion of nonlinear 
dynamical stability for barotropic stars described by the 
Euler-Poisson system (Sec. I2.1f) is similar to a criterion 
of thermodynamical stability in the canonical ensemble. 
Using this thermodynamical analogy, we provide a deriva- 
tion and an interpretation of the nonlinear Antonov first 
law (Sec. 14.2(1 in terms of "ensembles inequivalence": a 
spherical stellar system with / = /(e) and /'(e) < is 
nonlinearly dynamically stable with respect to the Vlasov- 
Poisson system if the corresponding barotropic star with 
the same equilibrium density distribution is nonlinearly 
dynamically stable with respect to the Euler-Poisson sys- 
tem. This is similar to the fact that "canonical stability 
implies microcanonical stability" in thermodynamics (see, 
e.g., Ellis et al. 2000, Bouchet & Barre 2005). The converse 
is wrong in case of "ensembles inequivalence" which is 
generic for systems with long-range interactions like grav- 
ity. We show that criteria of nonlinear dynamical stability 
can be obtained very simply from purely graphical con- 
structions by using the method of series of equilibria and 
the turning point argument of Poincare fSecs. rOl and ^.Sl) . 
like in thermodynamics (SecOJ). This will open a connec- 
tion between studies of dynamical and thermodynamical 
stability fSec. l4.4)l . These results were sketched at the end 
of a preceding paper (Chavanis 2003a) but we think that 
it is important to re-discuss them here in a more logical 
way with amplification. 

Optimization principles for self-gravitating systems 
have also been developed in connection with a notion 
of "generalized thermodynamics" (Tsallis 1988). It has 
been observed in many systems with long-range inter- 
actions (self-gravitating systems, two-dimensional turbu- 
lence, HMF model,...) that coherent structures (galaxies, 
jets and vortices, clusters,...) form spontaneously and per- 
sist for extremely long times (see Chavanis 2002c). Since 
these meta-equilibrium states, or quasi-stationary states 
(Q.S.S.), are not described by the Boltzmann distribution 
in general, some authors have suggested that they might 
be described by a generalized thermodynamics. In partic- 
ular, it has been proposed to replace the Boltzmann en- 
tropy S B [f] by the Tsallis entropy S q [f] =-^J(f q - 
f)drdv. In that respect, the coherent structures that 
quickly form in long-range systems (in the collisionless 
regime) are claimed to maximize the Tsallis entropy at 



fixed mass and energy (Boghosian 1996, Latora et al. 
2002). In the case of self-gravitating systems, this gener- 
alized thermodynamical interpretation has been defended 
by Taruya & Sakagami (2003). We think, however, that 
this approach stems from a mis-interpretation (Chavanis 
2003a, 2005b). Metaequilibrium states such as galaxies re- 
sult from an incomplete violent relaxation (Lynden-Bell 
1967) and are stable stationary solutions of the Vlasov 
equation on the coarse-grained scale. The Tsallis func- 
tional H q [f] = — ^rr J{f q — f)drdv, expressed as a func- 
tion of the coarse-grained DF /(r, v), is a particular H- 
function in the sense of Tremaine et al. (1986), not a ther- 
modynamical entropy. Its extremization at fixed mass and 
energy leads to stellar polytropes fSec. I4.3(l which are par- 
ticular stationary solutions of the Vlasov equation that 
form simple mathematical models of spherical stellar sys- 
tems. Furthermore, its maximization provides a condition 
of nonlinear dynamical stability for the Vlasov-Poisson 
system, not a condition of "generalized thermodynamical 
stability" . The resemblance of the criterion of nonlinear 
dynamical stability with a criterion of "generalized ther- 
modynamical stability" is essentially the mark of a ther- 
modynamical analogy (Chavanis 2003a). In fact, the previ- 
ous authors did not consider the Vlasov equation (which 
is capital in our interpretation) and tried to build a gen- 
eralized thermodynamics in order to deal with observed 
non-Boltzmannian quasi-stationary states (Q.S.S.). Now, 
considering the Vlasov equation, there is no fundamen- 
tal reason why the Q.S.S. resulting from incomplete vio- 
lent relaxation should be described by stellar polytropes 
(Tsallis distribution). These form just a particular class 
of steady solutions of the Vlasov equation. In practice, it 
is extremely difficult to predict the metaequilibrium state 
reached by the system in case of incomplete relaxation 
and we are led to construct dynamically stable station- 
ary solutions of the Vlasov equation so as to reproduce 
observations a posteriori (Binney & Tremaine 1987). The 
Vlasov equation admits an infinite number of stationary 
solutions and only stable solutions must be considered. For 
spherical systems with / = /(e) and /'(e) < 0, a powerful 
criterion of nonlinear dynamical stability is given by the 
above maximization principle. A connection with theories 
of incomplete violent relaxation that try to predict the 
DF reached by the system (Lynden-Bell 1967, Stiavelli & 
Bertin 1987, Chavanis et al. 1996) is discussed in Sec. 03 

2. Dynamical stability of stars and galaxies 

2.1. Nonlinear dynamical stability of barotropic stars 

We consider a self-gravitating gaseous medium (star) de- 
scribed by the Euler-Poisson system 

^+V-(pu) = 0, (1) 

^ + (u.V)u--ivp-V$, (2) 
or p 

A$ = AitGp. (3) 
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We shall restrict ourselves to the case of a barotropic gas 
for which the equation of state p = p(p) depends only 
on the density. With this assumption, the Euler-Poisson 
system is closed. The total energy of the fluid is 



W[p,u] = 



p§dr 



p-dr. 



(4) 



The first term is the work —p(p)d(l/ p) done in compress- 
ing it from infinite dilution, the second term is the grav- 
itational energy and the third term is the kinetic energy 
associated with the mean motion. 

It is straightforward to verify that the energy func- 
tional (0J is conserved by the Euler-Poisson system (VV = 
0). Therefore, a minimum of W at fixed mass determines 
a stationary solution of the Euler-Poisson system which is 
(formally) nonlinearly dynamically stable in the sense of 
Holm et al. (1985). Physically, this means that a small 
perturbation from the minimum remains close (in some 
norm) to the minimum. We arc led therefore to consider 
the minimization problem 



Min{W[p,u] \M[p] = M}. 



(5) 



Cancelling the first order variations of Eq. (0} at fixed 
mass, we obtain u = and the condition of hydrostatic 
equilibrium 



(6) 



Therefore, extrema of W correspond to stationary solu- 
tions of the Euler-Poisson system Q-©- Combining the 
condition of hydrostatic equilibrium © and the equation 
of state p = p(p), we get 



(7) 



so that p is a function of <E> that we note p — p(Q). 
Considering now the second order variations, the condi- 
tion of nonlinear dynamical stability is 



5 2 W = ~ / dpWdr 



P'(p) 
2p 



(Sp) 2 dr > 0, 



for all perturbations that conserve mass, i.e. J Sp dr = 0. 
We note that the second integral in Eq. (JSJ) can be written 
in a more conventional form by using p'(p)/p = — l//o'($) 
resulting from Eq. (JJJ. 

Let us consider explicitly some particular examples. 
For an isothermal gas p — p^^-, we find that 



A'e 



W = k B T 



p— dr. 



(9) 



(10) 



mm 2 J J 2 

On the other hand, for a polytropic gas p = Kp 1 , we have 



A 



7-1 



(11) 



W 



K 



J( p -r-p)dr+~J p$dr + J p^-dv. (12) 

For 7 — * 1, we recover the expressions (0 and <|10[1 of 
the isothermal gas. The minimization problem JSJ has 
been studied in Chavanis (2002a, b, 2003a) for an isother- 
mal and a polytropic equation of state (the system can 
be self-confined or confined by a box or by an exter- 
nal pressure). It is shown that the condition of non- 
linear dynamical stability coincides with the condition 
of linear dynamical stability (see also Appendix EJ. On 
the other hand, for isothermal systems, the energy func- 
tional W[p, 0] coincides with the Boltzmann free energy 
F B [p\ = E[p] - TS B [p] (sec Chavanis 2002a for details). 
Therefore, the condition of nonlinear dynamical stability 
with respect to the Euler-Poisson system is the same as 
the condition of thermodynamical stability in the canoni- 
cal ensemble. 



2.2. Nonlinear dynamical stability of stellar systems 

We consider a stellar system (galaxy) described by the 
Vlasov-Poisson system 



df 
dt 



df 
Or 



A$ = AttG / fdv 



(13) 



(14) 



where F = — V<j> is the self-consistent gravitational field 
produced by the stars. The Vlasov description assumes 
that the evolution of the system is encounterless. This is 
a very good approximation for most stellar systems be- 
cause the relaxation time is in general larger than the age 
of the universe by several orders of magnitude (Binney & 
Tremaine 1987). The Vlasov equation conserves the total 
mass M — J fdrdv, the total energy E — \ J fv 2 drdv + 
h J p&dr and an infinite class of functionals called the 
Casimirs that are defined by 1^ = J h(f)drdv for any 
function h. We shall be particularly concerned with a spe- 
cial class of Casimirs of the form 



(8) S[f] = - / C(f)drdv, 



(15) 



where C is an arbitrary convex function, i.e. C" > 0. Since 
S, M and E are conserved by the collisionless dynamics, 
the maximization problem 



Max{S[/] | E[f] = E,M[f] = M}, 



(16) 



determines a nonlinearly dynamically stable stationary so- 
lution of the Vlasov-Poisson system. We note that F[f] = 
E[f]—TS[f] , where T is an arbitrary positive constant (we 
shall explain later why we restrict T to positive values), 
is also conserved by the Vlasov equation. This is called a 
Casimir-Energy functional. Therefore, a minimum of F at 
fixed mass M also determines a nonlinearly dynamically 
stable stationary solution of the Vlasov-Poisson system. 
This criterion can be written 



Min {F[f} | M[f] = M}. 



(17) 
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Instead of minimizing F at fixed mass, it proves more 
convenient to maximize the functional J = S — (3E at 
fixed mass, where f3 — 1/T. We shall often work with 
the functional J in the following. Similar criteria of non- 
linear dynamical stability have been introduced by Ellis 
et al. (2002) in the context of two-dimensional hydrody- 
namics described by the 2D Euler-Poisson system. The 
optimization problem l|17(l corresponds to the formal non- 
linear dynamical stability criterion of Holm et al. (1985) 
and the optimization problem (|16[1 corresponds to the re- 
fined dynamical stability criterion of Ellis et al. (2002). 
These criteria are in general not equivalent for systems 
with long-range interactions, and this is similar to a no- 
tion of "ensembles inequivalence" in thermodynamics (see 
Sec. 14. H . The criterion of Ellis et al. (2002) refines the non- 
linear dynamical stability theorems of Arnold (1966) who 
first introduced the Casimir-Energy method in 2D hydro- 
dynamics. Inspired by these studies, we have applied the 
same criteria Ijl6|l and (|17|) to study the nonlinear dynam- 
ical stability of collisionless stellar systems described by 
the Vlasov-Poisson system (Chavanis 2003a, b, Chavanis & 
Sire 2004). Similar optimization principles have been in- 
troduced independently by Guo & Rein (1999,2001) in the 
mathematical literature. They first considered the mini- 
mization of a Casimir-Energy functional E — S at fixed 
mass M (Guo & Rein 1999) and later obtained a refined 
criterion of nonlinear dynamical stability by minimizing 
the energy E at fixed Casimir S and mass M (Guo & 
Rein 2001). The first criterion is equivalent to minimizing 
F at fixed mass (optimization problem 117JI ) and the sec- 
ond criterion is equivalent to maximizing S at fixed energy 
E and mass M (optimization problem dJ). 

Using Lagrange multipliers and writing respectively 



SS - /3SE - aSM = 0, 



or 



8 J - aSM = 0, 



(18) 



(19) 



the critical points of the variational problems l|16|) & n d 
ifTTjl are both given by 



C'{f) = -Pe-a, 



(20) 



where e = \ + $(r) is the energy of a particle by unit of 
mass. Since C is a monotonically increasing function of 
/, we can inverse this relation to obtain 



/ = F{0e + a) 



(21) 



where F (x) — (C") _1 (— x). Therefore, the critical points 
of the optimization problems (|16(1 and (|17|l are the same. 
They correspond to distribution functions of the form 
/ = /(e) which depend only on the energy. According 
to the Jeans theorem, these DF are particular station- 
ary solutions of the Vlasov-Poisson system which form a 
sub-class of spherical galaxies (the general DF of spheri- 
cal galaxies depends on energy e and angular momentum 
J = r x v or J = |r x v|). From Eq. l(27)|l . we have the 



identity /'(e) = —f3/C"(f) showing that /(e) is a mono- 
tonic function. Since /(e) should be decreasing at least 
for large energies, this imposes (3 = 1/T > 0. Then, we 
conclude that /'(e) < for all e. Inversely, any DF of the 
form / = /(e) with /'(e) < is a critical point of a certain 
Casimir S of the form l|15|) at fixed mass and energy. 

Furthermore, any DF of the form / = /(e) with 
/'(e) < which satisfies the optimization problems H16|l 
or (|17JI is nonlinearly dynamically stable with respect to 
the Vlasov-Poisson system. The condition of nonlinear dy- 
namical stability provided by the criterion 1|16|) is 



5* J 



J C"(f)^-drdv ~\P J SpS^dr < 0, 



5E = SM = 0, 



(22) 



and the condition of nonlinear dynamical stability pro- 
vided by the criterion l|17l) is 



S 2 J 



C"(f) [ -^dYdv- l -P / ,V<I»,/r-_ll. 



VSf 



SM = 0. 



(23) 



The first integral can be written in a more conventional 
form by using /'(e) = —f3/C"(f). We note the importance 
of the first order constraints in the criteria (1221) and (|23|) . 
If condition (|23|) is satisfied for all perturbations that con- 
serve mass, it is a fortiori satisfied for perturbations that 
conserve mass and energy. Therefore, condition (|23|l im- 
plies condition 122|) . but not the opposite. This is similar 
to the fact that canonical stability implies microcanonical 
stability in thermodynamics (but not the opposite). We 
conclude that the stability criterion (|16J) is more refined 
than the stability criterion l|17|) : if a collisionless stellar 
system with / = /(e) and /'(e) < satisfies (JTHl or (tT7l) . 
it is nonlinearly dynamically stable; however, if it does not 
satisfy (jl7|l , it can be nonlinearly dynamically stable pro- 
vided that it satisfies (|16(l . This means that we can "miss" 
stable solutions if we use just the optimization problem 
(TT7Jl to construct stable stationary solutions of the Vlasov- 
Poisson system of the form / = /(e) with /'(e) < 0. The 
problem (|16fl is richer and allows to construct a larger 
class of nonlinearly dynamically stable models of spherical 
galaxies. In particular, we shall see that stellar polytropes 
of index 3 < n < 5 satisfy the criterion (|16[) but not the 
criterion l|17|) . 



3. Thermodynamics of self-gravitating systems 
and series of equilibria 

3.1. Classical particles 

Optimization problems of the form l|16|) and (|17|> were 
first studied in relation with the thermodynamics of stellar 
systems (Antonov 1962, Lynden-Bell & Wood 1968) where 
S is the Boltzmann entropy 



s B [f] 



f f 

— In — drdv, 

m m 



(24) 
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which can be obtained by a combinatorial analysis 
(Ogorodnikov 1965). In this thermodynamical context, the 
criterion i|16[l is a criterion of microcanonical stability and 
the criterion (|17f) is a criterion of canonical stability where 
F B [f] = E[f] — TSb[J] is the Boltzmann free energy and 
T the thermodynamical temperature. The critical points 
of Sb or Fb (with appropriate constraints) correspond to 
the mean-held Maxwell-Boltzmann distribution 



(25) 



where $(r) is related to / via the Poisson equation l|14l) . 
The optimization problems (|16[) or l|17f) determine the 
most probable state of the iV-body system (statistical 
equilibrium state) resulting from a "collisional" relax- 
ation. The microcanonical ensemble (fixed energy) is ap- 
propriate to isolated Hamiltonian systems like stellar sys- 
tems. The relaxation towards statistical equilibrium is 
driven by two-body encounters (Binney & Tremaine 1987) 
and the collisional evolution of stellar systems is described 
by the gravitational Landau equation which increases the 
Boltzmann entropy Sb (H-theorem) at fixed mass M and 
energy E. Therefore, collisions (finite N effects) eventually 
single out the Boltzmann distribution among all possible 
stationary solutions of the Vlasov equation. The canonical 
ensemble (fixed temperature) is appropriate to dissipative 
systems in contact with a heat bath (of non-gravitational 
origin) imposing its temperature, like in the model of self- 
gravitating Brownian particles introduced by Chavanis 
et al. (2002). The evolution of self-gravitating Brownian 
particles is described by mean-field Fokker-Planck equa- 
tions (Kramers and Smoluchowski) which decrease the 
Boltzmann free energy F B at fixed mass M (Chavanis 
2006b). 

As is well-known, there is no global maximum of Sb 
(resp. global minimum of Fb) but there can exist local 
maxima of Sb (resp. local minima of Fb) if the sys- 
tem is confined within a box of radius R and the energy 
(resp. temperature) is sufficiently high. These metastable 
states are long-lived because their lifetime scales as e N so 
that they are of prime interest in astrophysics (Chavanis 
2005a) . The thermodynamical stability of the system can 
be settled by using the turning point method of Poincare 
which was applied by Lynden-Bell & Wood (1968) and 
Katz (1978) to classical self-gravitating systems. Since we 
shall apply this method in new situations (related to non- 
linear dynamical stability problems), we first describe how 
this method works in thermodynamics referring to the pre- 
viously quoted papers for technical details. 

Suppose we want to maximize the Boltzmann en- 
tropy Sb at fixed mass M and energy E (microcanoni- 
cal stability). The parameter conjugate to the entropy Sb 
with respect to the energy E is the inverse temperature 
ft — (8Sb /9E) jv,v • Therefore, if we plot the series of equi- 
libria giving (3 as a function of —E, we have the following 
results: 

(i) a change of stability can occur only at a turning 
point where E is an extremum (d(3/dE infinite). 
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Fig. 1. Series of equilibria of isothermal stellar systems 
described by the Boltzmann entropy. The relevant di- 
mensionless energy and dimensionless temperature are 
A = — and rj — n™ ■ The thermodynamic limit 
N — > +oo is such that these quantities remain fixed 
(Chavanis 2006b). In the microcanonical ensemble, the 
stable states (local entropy maxima LEM) are located on 
the full curve (microcanonical caloric curve). After the 
turning point of energy MCE, the states become unstable 
(saddle points SP of entropy). 



(ii) a mode of stability is lost if the curve rotates clock- 
wise and gained if the curve rotates anti-clockwise. 

The series of equilibria of isothermal stellar systems 
described by the Boltzmann entropy is shown in Fig. ^ 
and forms a spiral. Now, we know that for E sufficiently 
large, the solutions are stable (maxima of Sb at fixed E, 
M) because, in this limit, self-gravity is negligible and 
the system behaves like an ordinary gas in a box. From 
point (i), we conclude that the whole upper branch un- 
til point MCE is stable and corresponds to local entropy 
maxima (LEM). This stable part of the curve defines the 
microcanonical caloric curve. At the first turning point of 
entropy MCE, the curve rotates clockwise so that a mode 
of stability is lost. As the curve spirals inward, more and 
more modes of stability are lost (at each vertical tangent) 
so that the states past MCE are unstable saddle points of 
entropy (SP). 

Suppose now that we want to minimize the free energy 
F at fixed mass (canonical stability). This is equivalent to 
maximizing the Massieu function J = S — f3E at fixed M. 
Now the parameter conjugate to J with respect to /3 is 
— E = {dJ/d(3)N,v- Therefore, by using the turning point 
criterion (properly adapted to the present case) 1 we con- 
clude that the upper branch of the spiral is stable in the 
canonical ensemble until the point CE and corresponds 
to local minima of free energy (FEM). This stable part 
of the curve defines the canonical caloric curve. At the 



1 We just need to rotate Fig. |5|by 90° and plot E as a func- 
tion of j3 to be in the conditions of application of (i) and (ii). 
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Fig. 2. Scries of equilibria of isothermal stellar systems 
described by the Boltzmann free energy. In the canoni- 
cal ensemble, the stable states (local free energy minima 
LFEM) are located on the full curve (canonical caloric 
curve). After the turning point of temperature CE, the 
states become unstable (saddle points SP of free energy). 
The region of negative specific heats between CE and 
MCE is a domain of ensembles inequivalence: those states 
are stable in the microcanonical ensemble but unstable in 
the canonical ensemble. 



first turning point of temperature CE, the curve rotates 
clockwise so that a mode of stability is lost. As the curve 
spirals inward after CE, more and more modes of stability 
are lost (at each horizontal tangent) so that the states are 
saddle points of free energy (SP). We note that the states 
between CE and MCE are stable in the microcanonical 
ensemble (local maxima of entropy Sb at fixed E, M) but 
unstable in the canonical ensemble (saddle points of free 
energy Fb at fixed M). Such states have negative specific 
heats C = (dE j 'dT) n ,v ■ Negative specific heats are for- 
bidden in the canonical ensemble but they are allowed in 
the microcanonical ensemble for systems with long-range 
interactions for which the energy is non-additive (Thirring 
1970, Lynden-Bell & Lynden-Bell 1977). Therefore, in this 
region, the ensembles are inequivalent: the states with neg- 
ative specific heats cannot be reached by a canonical de- 
scription (they are unstable). Note that negative specific 
heats is a sufficient but not a necessary condition of canon- 
ical instability: for example, the states between MCE and 
CE2 have positive specific heats but they are canonically 
(and microcanonically) unstable. 



3.2. Self-gravitating fermions 

Another application of the turning point method where 
modes of stability can be lost and then re-gained has been 
studied by Chavanis (2002d) in the case of self-gravitating 
fermions. In that case, the thermodynamical potential is 



Fig. 3. Series of equilibria of self-gravitating fermions de- 
scribed by the Fermi-Dirac entropy for /i = 10 5 . In the 
microcanonical ensemble, the stable states (entropy max- 
ima EM) are located on the full curves. The upper branch 
corresponds to the "gaseous phase" and the lower branch 
to the "condensed phase" . The mode of stability lost at 
MCE1 is re-gained at MCE2. The dashed curve corre- 
sponds to unstable saddle points of entropy. 
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Fig. 4. Series of equilibria of self-gravitating fermions de- 
scribed by the Fermi-Dirac entropy for /j, = 10 3 . In the mi- 
crocanonical ensemble, the full series of equilibria is stable 
and corresponds to global entropy maxima (GEM). 



the Fermi-Dirac entropy 



in 



Vo Vo 



no 



drdv, 



(26) 



and its extremization at fixed mass and energy leads to 
the Fermi-Dirac distribution 



/ = 



Vo 



1 + \ e P™(^+Hr)) ' 



(27) 
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Fig. 5. Scries of equilibria of self-gravitating fermions de- 
scribed by the Fermi-Dirac free energy for /_t = 10 3 . In the 
canonical ensemble, the stable states (minima of free en- 
ergy FEM) are located on the full curves. The left branch 
corresponds to the "gaseous phase" and the right branch 
to the "condensed phase". The mode of stability lost at 
CE1 is re-gained at CE2. The dashed curve corresponds 
to unstable saddle points of free energy. 



where A > and r]o = (m//i) 3 is the maximum allowed 
value of the distribution function due to the Pauli exclu- 
sion principle. The form of the scries of equilibria depends 
on the degeneracy parameter /1 = ?7o\/5127r 4 G 3 A/i? 3 2 . 
For a degeneracy parameter — 10 5 , the series of equilib- 
ria is shown in Fig. [3] The stability of the configurations 
can be analyzed with the Poincare turning point argu- 
ment. For large E, self-gravity is negligible with respect to 
thermal motion and the system coincides with a classical 
perfect gas in a box which is known to be thermodynami- 
cally stable. Thus, from (i) we conclude that the series of 
equilibria is microcanonically stable until MCE1. At that 
point, the tangent is vertical and the curve rotates clock- 
wise so that a mode of stability is lost. However, a second 
turning point of energy appears at MCE2 and the curve 
turns anti-clockwise so that the mode of stability is re- 
gained. Therefore, the states between MCE1 and MCE2 
are unstable saddle points of entropy at fixed mass and en- 
ergy (SP) while the states before MCE1 and after MCE2 
are maxima of entropy at fixed E, M. These stable states 
define the physical microcanonical caloric curve including 

2 The degeneracy parameter can also be written /i = 
17.259...(i?/-R») 3,/2 . It is proportional to the ratio, to the power 
3/2, between the system size R and the size R, of a completely 
degenerate configuration (fermion ball at T = 0) with mass M 
(Chavanis & Rieutord 2003). Roughly speaking, it is a mea- 
sure of the importance of the small-scale regularization which 
is played here by the Pauli exclusion principle. Thus, e = 1/fi 
can be seen as an effective small-scale cut-off. In terms of the 
Planck constant, we have l//_t ~ Tl 3 so that the classical limit 
(h — > 0) is recovered for /1 —> +00. 



the metastable states (local entropy maxima). The transi- 
tion between local and global entropy maxima, the impor- 
tance of metastable states and the notion of phase tran- 
sitions (first order and zeroth order) between a gaseous 
phase and a condensed phase in the self-gravitating Fermi 
gas, and more generally in self-gravitating systems with 
a small-scale cut-off, are further discussed in Chavanis 
(2002d,2005a). 

For a degeneracy parameter /i = 10 3 , the series of equi- 
libria is shown in Figs. 0] and [S] Since there is no turning 
point of energy, we conclude that the whole series of equi- 
libria is stable in the microcanonical ensemble and corre- 
sponds to global entropy maxima (GEM) at fixed mass 
and energy. This forms the microcanonical caloric curve 
(see Fig. 0}. Alternatively, in the canonical ensemble (see 
Fig. EJ), the presence of turning points of temperature and 
the application of the Poincare method (using the fact 
that for high temperatures the system is stable) implies 
that the series of equilibria is stable before CE1 and after 
CE2 (minima of free energy at fixed mass) and unstable in 
between (saddle points of free energy at fixed mass). The 
mode of stability lost at CE1 is re-gained at CE2. The 
stable states on the solid curves form the physical canoni- 
cal caloric curve including the metastable states. By com- 
paring Figs. 0] and [5] we note that the region of negative 
specific heats C < in the microcanonical ensemble is in- 
accessible (unstable) in the canonical ensemble where it is 
replaced by a phase transition. 

Note, finally, that the Fermi-Dirac distribution l|27|l 
also arises in the theory of violent relaxation for colli- 
sionless stellar systems (Lynden-Bell 1967, Chavanis & 
Sommeria 1998b). In that case, the exclusion principle is 
of dynamical origin since it is related to the conservation of 
the fine-grained DF by the Vlasov equation. Furthermore, 
the maximum phase-space density 770 is related to the ini- 
tial condition while 770 ~ h~ 3 for fermions. Degeneracy 
effects (in the sense of Lynden-Bell) may be important in 
numerical simulations of violent relaxation (May & van 
Albada 1984) and, possibly, in the core of elliptical galax- 
ies (Stiavelli & Bertin 1987, Chavanis & Sommeria 1998b) 
and dark matter (Kull et al. 1996). 

4. Dynamical stability of stellar systems and 
barotropic stars 

4.1. Thermodynamical analogy 

Formally, the criteria of nonlinear dynamical stability l|16f) 
and (|17|) for "collisionless" stellar systems are similar to 
criteria of thermodynamical stability for "collisional" self- 
gravitating systems (Sec. |3} but they involve a more gen- 
eral functional S[f] = — J C(f)dvdv than the Boltzmann 
entropy Ss[f] = — f In -^drdv. Furthermore, they have 
a completely different interpretation as they provide cri- 
teria of nonlinear dynamical stability for a stationary so- 
lution of the Vlasov equation while the maximization of 
the Boltzmann entropy at fixed mass and energy provides 
a condition of thermodynamical stability (most probable 
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state) for the statistical equilibrium state of a iV-body 
stellar system reached for t — > +00. 

Due to the formal resemblance between nonlinear dy- 
namical stability criteria and thermodynamical stability 
criteria, we can develop a thermodynamical analogy to in- 
vestigate the nonlinear dynamical stability of collisionless 
stellar systems and use a vocabulary borrowed from ther- 
modynamics. In this analogy, S is similar to an entropy, 
T = X/(3 is similar to a temperature and F is similar 
to a free energy (it is related to S by a Legendre trans- 
form). The criterion 1)16(1 is similar to a condition of mi- 
crocanonical stability and the criterion 1)17(1 is similar to 
a condition of canonical stability. Since canonical stability 
implies microcanonical stability (but not the converse), 
we recover the fact that condition 1(23(1 implies condition 
(JUJ). The stability problems JTJJ and JT7J can be stud- 
ied by plotting the linear series of equilibria (3(E), which 
is similar to a caloric curve in thermodynamics. Due to 
the above thermodynamical analogy, the method of lin- 
ear series of equilibria can be used to settle the nonlinear 
dynamical stability of a collisionless stellar system for a 
general functional S. According to the Poincare theorem, a 
mode of stability is lost or gained at a turning point. Note 
that the slope dE/dT (where (3 = 1/T is the Lagrange 
multiplier associated with the energy) plays an important 
role because, according to the turning point argument, 
its change of sign may be the signal of an instability. The 
slope C = dE/dT is similar to the specific heat in thermo- 
dynamics. We shall give some illustrations of these results 
in the following sections. 

4.2. The nonlinear Antonov first law 

For any stellar system with / = /(e) and /'(e) < 0, there 
exists a corresponding barotropic star with the same equi- 
librium density distribution. Indeed, defining the density 
and the pressure by p = J fdv — p(&), p = | / fv 2 dv = 
p($), and eliminating the potential $ between these two 
expressions, we find that p = p(p). Writing explicitly the 
density and the pressure in the form 

r+00 

p = Aix /(e)V2(e-*)de, (28) 

a r+oo 

P=YJ^ /( e )[2(e-*)] 3/a &, (29) 

and taking the gradient of Eq. 1(29(1 , we obtain the condi- 
tion of hydrostatic equilibrium 10. 

As discussed in Sec. 12.21 the minimization problem 
l(17|) provides a sufficient (but not necessary) condition 
of nonlinear dynamical stability for a collisionless stel- 
lar system. To solve this minimization problem, we can 
first minimize F[f] at fixed density p(r) to obtain /*(r, v). 
More precisely, /* is determined by C"(/«) = —/3(v 2 /2 + 
A(r)) where A(r) is the Lagrange multiplier associated 
with the specification of the density p(r). Since 5 2 F = 
(T/2) J C"(f*)(6f) 2 drdv > 0, /* really is a minimum of 
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Fig. 6. Series of equilibria for polytropic spheres with in- 
dex n = 2 (see Chavanis 2002b and Chavanis & Sire 2004 
for the details of the construction) . In the thermodynam- 
ical analogy, this can be viewed as an effective caloric 
curve giving the effective normalized inverse temperature 
T) = M(G/nS)^/R^ (where n@ = K(l + n)/(4 7 r) 1 /" 
is a polytropic temperature proportional to the poly- 
tropic constant K) as a function of the normalized en- 
ergy A = —ER/GM 2 . The parameter rj is a monotonic 
function of the Lagrange multiplier (3 (through the re- 
lation K = /3 _ ( 1- &) obtained from Eqs. (24) and (31) 
of Chavanis & Sire 2004). Therefore, the curve 77 — A is 
equivalent, with more convenient variables, to the curve 
(3(E) issued from the optimization problems 1(16(1 and l(17|). 
The first complete polytrope, with p(R) = 0, is indicated 
by (•). It corresponds to the terminal point in the se- 
ries of equilibria. The branch of complete polytropes with 
i?„ < R is indicated by a dashed line. For n < 3, the 
series of equilibria is monotonic so that the "ensembles" 
are equivalent. Due to the correspondence that we have 
found, this means that for n < 3 both stellar polytropes 
and polytropic stars are nonlinearly dynamically stable 
with respect to the Vlasov-Poisson and Euler-Poisson sys- 
tems respectively. 



F at fixed p(r) (note that fixing p(r) automatically de- 
termines the potential energy W — (1/2) J p&dr which 
can thus be treated as a constant) . From the optimal dis- 
tribution function /*, we can compute the density p = 
J /*dv = p(X) and the pressure p = | f f*v 2 dv = p(X). 
Eliminating A(r) between these two expressions, we ob- 
tain the equation of state of a barotropic gas p = p(p). 
This is the same equation of state as in equilibrium where 
A(r) = <I>(r) + ctj (3 but the important point is that the 
relation p = p(p) also holds for perturbations dp around 
equilibrium. We can then express the functional F[f] as a 
functional of the density p(r) by setting F[p] — F[f*\. The 
calculations are detailed in Appendix [5] (see also Sees. 7.7 
and 7.8 of Chavanis 2003a) and the functional F[p] can be 
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finally written 



1.2 



F[P\ = \j P^dr + J p J 



(30) 



where p(p) is the equation of state of the corresponding 
barotropic gas defined above. We are thus led to consider 
the minimization problem 



Min {F[p} | M[p] = M}. 



(31) 



Now, we observe that the functional (|30|l coincides with 
the energy functional 10} of a barotropic gas with u = 0. 
We come therefore to the following conclusion: for a given 
stellar system with / = /(e) and /'(e) < 0, if we know 
that the corresponding barotropic star is nonlinearly dy- 
namically stable with respect to the Euler-Poisson system, 
then it is a minimum of W[p, u], hence of F[p] (at fixed 
mass). Therefore, the stellar system satisfies the criterion 
l|17l) so it is nonlinearly dynamically stable with respect to 
the Vlasov-Poisson system. This leads to a nonlinear gen- 
eralization of the Antonov first law 3 : "a spherical stellar 
system with / = /(e) and /'(e) < is nonlinearly dynam- 
ically stable with respect to the Vlasov-Poisson system if 
the corresponding barotropic gas with the same equilib- 
rium density distribution is nonlinearly dynamically stable 
with respect to the Euler-Poisson system" . However, the 
reciprocal is wrong in general. A stellar system can be non- 
linearly dynamically stable according to the "microcanon- 
ical" criterion (|16fl while it does not satisfy the "canonical" 
criterion (|17J) so that the corresponding barotropic star is 
dynamically unstable. Due to the thermodynamical anal- 
ogy, the nonlinear Antonov first law for collisionlcss stellar 
systems has the same status as the fact that: "canoni- 
cal stability implies microcanonical stability in thermody- 
namics" . The converse is wrong in case of "ensembles in- 
equivalence" which is generic for systems with long-range 
interaction such as gravity. To the point of view of their 
nonlinear dynamical stability, the crucial difference be- 
tween stars and galaxies is that, for galaxies, both S and 
E are individually conserved by the Vlasov equation while 
in the case of stars only the total energy W is conserved 
by the Euler equations. Therefore, the most refined stabil- 
ity criterion for spherical galaxies is 1(16|1 while the most 
refined stability criterion for stars is JSJ, which is included 
in (|16fl . This is the intrinsic reason why a spherical galaxy 
can be stable even if the corresponding barotropic star is 
unstable. Therefore, the set of stable spherical galaxies is 
larger than the set of stable barotropic stars. Similarly, in 
thermodynamics, the set of stable microcanonical equilib- 
ria is wider than the set of stable canonical equilibria (it 
includes in particular states with negative specific heats) 
because there is an additional constraint in the problem: 
the energy. 
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Fig. 7. Series of equilibria for polytropic spheres with in- 
dex n = 4. The first complete polytrope, with i?* = R, 
is indicated by (•) and the branch of complete polytropes 
with i?* < R by a dashed line. Since they lie after the 
turning point of "temperature" (appearing for n — 3) and 
before the turning point of energy (appearing for n = 5), 
in the thermodynamical analogy, they are unstable in the 
"canonical" ensemble (saddle points of "free energy" F 
at fixed M) but stable in the "microcanonical" ensemble 
(maxima of "entropy" S at fixed E,M). Due to the cor- 
respondance that we have found (see Sec. 14.2(1 . these ef- 
fective thermodynamical stability criteria mean in reality 
that stellar polytropes with n = 4 are nonlinearly dynami- 
cally stable stationary solutions of the Vlasov-Poisson sys- 
tem but polytropic stars with n = 4 are linearly dynam- 
ically unstable stationary solutions of the Euler-Poisson 
system. This is similar to a situation of "ensembles in- 
equivalence" in thermodynamics. In particular, complete 
polytropes with n = 4 lie in a region of effective negative 
specific heats C < 0. 

4.3. Application to polytropes 

It is interesting to consider the application of these results 
to the case of polytropes. Stellar polytropes extremize the 
Casimir functional 

1 



1 



(/« - f)drdv, 



(32) 



at fixed mass and energy. This leads to the class of distri- 
bution functions 



/ = 



P 



(33) 



with the convention that / = when the term in brackets 
becomes negative. The usual polytropic index n is related 
to the parameter q by n = | + — zj ■ We can now plot the se- 
ries of equilibria (3(E) issued from the optimization prob- 
lems <|16[) and l|17fl with the polytropic functional (|32|) . 

3 This result has been obtained independently in Chavanis 
(2003a) and Rein (2003,2005). 
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Fig. 8. Series of equilibria for polytropic spheres with in- 
dex n — 10. For n > 5, self-gravitating polytropic spheres 
are unbounded and have infinite mass. Thus, equilibrium 
distributions only exist if the system is confined within a 
box. Box-confined gaseous polytropes cease to be stable 
with respect to the Euler-Poisson system after the first 
turning point of "temperature" n and box-confined stellar 
polytropes cease to be stable with respect to the Vlasov- 
Poisson system after the first turning point of energy A. 
These stability criteria can be expressed in terms of the 
density contrast between center and edge which parame- 
terizes the series of equilibria, like for isothermal spheres. 

The equations determining these curves (for various index 
n) have been given in previous papers (Chavanis 2003a, 
Chavanis & Sire 2004) so we shall only describe the re- 
sults and present a "gallery" of caloric curves. We recall 
that the system is confined within a box of radius R so 
that the series of equilibria is essentially formed by incom- 
plete polytropes (represented in solid line) which are held 
by the box so that p(R) > 0. The series of equilibria can 
be parameterized by the density contrast TZ — p(0)/p(R) 
between center and edge. The terminal point of the series 
of equilibria corresponds to the first complete polytrope 
(represented by a bullet) for which the density drops to 
zero precisely at the box radius p(R) = (if n < 5). In the 
present context, this is the most important point in the 
series of equilibria because it connects the branch of in- 
complete polytropes to the branch of complete polytropes 
(represented by a dashed line) for which the density drops 
to zero at i?„ < R (if n < 5). These are the physical 
structures in astrophysics since they are not affected by 
an artificial box. We thus have to investigate the stability 
of the first complete polytrope (bullet) to determine the 
stability of all the complete polytropes 4 which are of phys- 
ical relevance. To that purpose, we just need to investigate 
the presence of turning points in the series of equilibria. 

4 In fact, complete polytropes are equivalent by the homol- 
ogy theorem (Chandrasekhar 1942) but this is not the case in 
general for other stellar systems. 



Therefore, our method is very simple and very general. It 
just requires to solve the first order variations of the opti- 
mization problems l|ltjfl and (|17fl and use the turning point 
argument to obtain powerful results of nonlinear dynami- 
cal stability for barotropic stars and spherical galaxies. 

For n < 3 the series of equilibria /3(E) for polytropes is 
monotonic (see Fig. so that, according to the Poincare 
criterion, all the polytropes of the series are maxima of 
S (at fixed E, M) and minima of F (at fixed M). In 
our thermodynamical analogy, this corresponds to a situa- 
tion of ensemble equivalence. These results imply that, for 
n < 3, both gaseous polytropes and stellar polytropes are 
nonlinearly dynamically stable with respect to the Euler- 
Poisson and Vlasov-Poisson systems respectively. It can 
be shown that a turning point of "temperature" (3 ap- 
pears for n > 3 (Chavanis 2002b). Therefore, complete 
polytropes (and incomplete polytropes with high density 
contrast lying after the turning point of "temperature") 
cease to be minima of F (at fixed mass) for n > 3 (see 
Fig. EJ. In that case, they lie in a region of the series of 
equilibria where the slope dE/dT is negative. In the ther- 
modynamical analogy, this is similar to a loss of canonical 
stability in a region of negative specific heats. Using the 
preceding results, we conclude that complete polytropic 
stars are nonlinearly dynamically stable with respect to 
the Euler-Poisson system for n < 3 but they become lin- 
early dynamically unstable for n > 3. On the other hand, 
a turning point of energy E appears for n > 5 (see Fig.[SJ). 
Therefore, complete polytropes are maxima of S (at fixed 
mass and energy) for n < 5. Polytropes cease to be self- 
confined for n > 5 and incomplete polytropes with high 
density contrast lying after the turning point of energy be- 
come unstable with respect to the Vlasov-Poisson system 
for n > 5. This is similar to a loss of microcanonical stabil- 
ity in the thermodynamical analogy. In conclusion, stellar 
polytropes are nonlinearly dynamically stable with respect 
to the Vlasov-Poisson system for 3/2 < n < 5 (they do not 
exist for n < 3/2) and they become dynamically unstable 
for n > 5 above a critical density contrast. On the other 
hand, gaseous polytropes are nonlinearly dynamically sta- 
ble with respect to the Euler-Poisson system for < n < 3 
and they become dynamically unstable for n > 3 above 
a critical density contrast. For 3 < n < 5, complete stel- 
lar polytropes are nonlinearly dynamically stable with re- 
spect to the Vlasov-Poisson system while corresponding 
polytropic stars are dynamically unstable with respect to 
the Euler-Poisson system. This is similar to a situation of 
ensembles inequivalence, in a region of negative specific 
heats, in thermodynamics. Of course, the dynamical sta- 
bility of stellar polytropes and polytropic stars has been 
established for a long time in astrophysics but our method 
relying on the appearance of turning points in the series 
of equilibria and the interpretation of these results, and 
of the Antonov first law, in terms of a thermodynamical 
analogy and a situation of ensembles inequivalence is new 
and original. Furthermore, only linear dynamical stability 
is considered in general while the criteria (|16J) and Ijl7|l 
are criteria of nonlinear dynamical stability. 
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4.4. The main point: corresponds nces 

In summary, we have the following analogy. In thermody- 
namics, if we plot the series of equilibria (3(E) we have the 
following result: 

• After a turning point of temperature (or dimension- 
less parameter n) , the system becomes canonically unsta- 
ble (saddle point of free energy F at fixed M). After 
a turning point of energy (or dimensionless parameter 
A) , the system becomes microcanonically unstable (saddle 
point of entropy S at fixed E, M) . If the series of equilibria 
does not unwind, the system remains thermodynamically 
unstable after these turning points. Before these turning 
points, the system is thermodynamically stable. 

Concerning the nonlinear dynamical stability of col- 
lisionless stellar systems with / = /(e), /'(e) < and 
corresponding barotropic stars with the same density dis- 
tribution, if we plot the series of equilibria /3(E) we have 
the following result: 

• After a turning point of "temperature" f3 (or dimen- 
sionless parameter 77) , the barotropic gas becomes dynam- 
ically unstable with respect to the Euler-Poisson system. 
After a turning point of energy E (or dimensionless pa- 
rameter A), the stellar system becomes dynamically un- 
stable with respect to the Vlasov-Poisson system. If the 
series of equilibria does not unwind, the system remains 
unstable after these turning points. Before these turning 
points, the system is nonlinearly dynamically stable. 

Schematically, we have the correspondances: 

(i) Nonlinear dynamical stability of 
barotropic stars <-> ^canonical" criterion 
(ii) Nonlinear dynamical stability of spherical 
stellar systems <-> ll microcanonical" criterion 
(Hi) Nonlinear Antonov first law 
<-> canonical stability implies microcanonical stability 

4.5. Other dimensions of space 

For completeness, we also show the series of equilibria 
for polytropic spheres in other dimensions of space d. 
These results can be of interest for mathematicians (see, 
e.g., Lemou et al. 2005). Furthermore, the problem is rich 
and interesting because it exhibits several critical dimen- 
sions with intriguing consequences concerning the partic- 
ular dimension of our universe (see Chavanis 2004). The 
construction of these curves is detailed in Chavanis & 
Sire (2004) and some new curves are reported here in 
order to have a complete picture of the problem in pa- 
rameter space (d,n). We first recall that physical stellar 
polytropes (whose DF is a decreasing function of the en- 
ergy) only exist for n > n 3 / 2 = d/2 while gaseous poly- 
tropes can be considered a priori for any index (but we 
shall mostly consider the case n > here). Furthermore, 
complete polytropes exist for all index if d < 2 and for 
n < n 5 = (d + 2)/(d— 2) if d > 2. Here, we only describe 



the stability of complete polytropes, i.e. the terminal point 
(bullet) in the series of equilibria. 

For d = 1 and d = 2 the series of equilibria are mono- 
tonic (see Figs. 191 lfl|) implying that both stellar polytropes 
with n > n 3 /2 and polytropic stars with n > are nonlin- 
early dynamically stable (the "ensembles" are equivalent). 

For 2 < d < 4, complete stellar polytropes exist for 
n 3/2 < n < n$ and they are nonlinearly dynamically sta- 
ble (the turning point of energy appears for n > n§). On 
the other hand, complete polytropic stars arc nonlinearly 
dynamically stable for < n < 713 = d/(d — 2) (713 is 
the index at which the turning point of "temperature" 
appears) and unstable for n$ < n < n§ (see Figs. Ill|) . 
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Fig. 9. Series of equilibria for polytropic spheres in d — 1. 
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Fig. 10. Series of equilibria for polytropic spheres in d = 
2. 

For d = 4, complete stellar polytropes exist for n 3 / 2 = 
2 < n < ri5 = 3 and they all have energy E — (see 
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Fig. 11. Series of equilibria for polytropic spheres in d = 
3. 

Fig- El as can be deduced from the results of Chavanis & 
Sire (2004) . The series of equilibria has a vertical tangent 
at E — implying that complete stellar polytropes are 
marginally stable in d — 4 (they lie precisely at the turn- 
ing point of energy). This is consistent with the Virial 
theorem in d = 4 for an isolated system (see Appendix 
C of Chavanis & Sire 2005). On the other hand, com- 
plete polytropic stars are nonlinearly dynamically stable 
for < n < ri3 — 2 and unstable for 773 = 2 < n < 775 = 3 
due to the presence of the turning point of "temperature" . 

For d > 5, there are no complete stellar polytropes 
with n > 7J3/2 (the cross-over between 715 and 713/2 takes 
place for d — 2(l+-\/2) ~ 4.83...). On the other hand, com- 
plete polytropic stars are nonlinearly dynamically stable 
for n < 113 and unstable for 713 < 71 < 775 due to the pres- 
ence of the turning point of "temperature" (see Fig. I13|) . 
For 77, = 77,3/2 the series of equilibria makes angular points 
as explained in Appendix C of Chavanis & Sire (2004). 
For 77 > 775, the series of equilibria forms a spiral whose 
sense of rotation changes for 77 = 773/2- For d > 10 the sit- 
uation slightly changes in the sense that the spiral shrinks 
to a point after an index n_ (see Chavanis & Sire 2004). 
It is of interest to note that the spatial dimension d = 10 
is special in classical gravity (see Sire & Chavanis 2002) 
as also found in theories of supergravity (e.g. Cariglia & 
Mac Conamhna 2005). We do not know whether this is a 
coincidence of if it bears more physical meaning. 

We note finally that classical white dwarf stars are 
equivalent to polytropes with index 7*3/2 = d/2 (Chavanis 
& Sire 2004). Therefore, they are self-confined (complete) 
if 773/2 < 775, i.e. d < 2(1 + V2) — 4.83... and they are sta- 
ble if 773/2 < 773, i.e. d < 4. For d > 4, quantum mechanics 
(Pauli exclusion principle) cannot stabilize matter against 
gravitational collapse (Chavanis 2004). Relativistic white 
dwarf stars are equivalent to polytropes with index n 3 = d. 
Therefore, they are self-confined (complete) if n' 3 < 775, i.e. 
d < ^(3 + \/T7) = 3.56... and they are stable if n' 3 < 773, 
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Fig. 12. Series of equilibria for polytropic spheres in d = 
4. 
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Fig. 13. Scries of equilibria for polytropic spheres in d = 
5. For clarity, the curves have been multiplied by some 
factor to enter in the same figure. 

i.e. d < 3. These results will be discussed more specifically 
in another paper (Chavanis 2006c). 

5. Phenomenology of violent relaxation 

In this section we show that the criterion of nonlinear 
dynamical stability Ijl6|) is remarkably consistent with 
the phenomenology of violent relaxation. Collisionless 
stellar systems like elliptical galaxies are believed to 
have achieved a steady state as a result of phase mix- 
ing and incomplete violent relaxation (Lynden-Bell 1967). 
During this process, the fine-grained distribution function 
/(r, v, t) develops intermingled filaments at smaller and 
smaller scales so that it does not converge toward any 
steady distribution. However, if we locally average over 
these filaments, the coarse-grained distribution function 
f(r,v,t) is expected to converge towards a steady state 
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/(r, v) which is a stable stationary solution of the Vlasov 
equation. This is referred to as weak convergence in mathe- 
matics. Thus, the theory of violent relaxation explains how 
a collisionless stellar system, initially out-of-equilibrium, 
can reach a steady state of the Vlasov equation (on a 
coarse-grained scale) due to mean-field effects. Since this 
metaequilibrium state results from a complicated mixing 
process it is expected to be particularly robust and have 
nonlinear dynamical stability properties. 

Now, Tremaine et al. (1986) have shown that the func- 
tionals S[f] = — J C(f )drdv 1 where C is convex, cal- 
culated with the coarse-grained distribution function in- 
crease during violent relaxation in the sense that S(t) > 
5(0) for t > where it is assumed that the DF is not 
mixed at t = 0, i.e. / (r, v, 0) = /(r, v, 0). For this reason, 
these functionals are called generalized ii-functions. By 
contrast, the energy E[f] and the mass M[f] calculated 
with the coarse-grained DF are approximately conserved. 
The fact that S[f] increases at fixed E[f] and M[f] is sim- 
ilar to a generalized selective decay principle (for —5), like 
in 2D turbulence, which is due to phase mixing and coarse- 
graining. Phenomenologically, this may suggest that the 
system will reach a steady state /(r, v) which maximizes 
a certain ff-function (non-universal) at fixed mass and 
energy. This is consistent with the criterion of nonlin- 
ear dynamical stability l|16[l provided that we interpret 
the distribution function as the coarse-grained distribu- 
tion function. Phase mixing and coarse-graining explain 
why we have to maximize a certain Casimir functional al- 
though the evolution is dissipationless and the Casimirs 
are conserved on the fine-grained scale 5 . 

In reality, the problem is more complicated than that 
because (i) the ii-functions do not increase necessarily 
monotonically with time (they are not Lyapunov function- 
als satisfying 5 > 0) (ii) the system can converge toward 
a stationary solution of the Vlasov equation which does 
not depend on the energy e alone, and thus which does 
not maximize an ff-function at fixed E, M. For exam- 
ple, the velocity distribution of stars in elliptical galax- 
ies is anisotropic and depends on the angular momentum 
J = |r x v| in addition to energy e. Furthermore, real 
stellar systems are in general not spherically symmetric 
so their DF does not only depend on e and J. Therefore, 
more general stationary solutions of the Vlasov equation 
must be constructed in consistency with the Jeans theo- 
rem (Binney & Tremaine 1987). However, their dynamical 
stability is more complicated to investigate. 

More than constructing particular stationary solutions 
of the Vlasov equation in order to reproduce observations, 



5 The physical reason is the following. During mixing 
Df/Dt ^ and the //-functions S[f] increase. Once it has 
mixed, Df/Dt — so that S[f] = meaning that the Casimirs 
of the coarse-grained DF are now conserved. If f(r, v, t) has 
been brought to a maximum / (r, v) of a certain H- function 
(during violent relaxation) and since S[f] is conserved (after 
mixing), then / is a nonlinearly dynamically stable steady 
state of the Vlasov equation against coarse-grained perturba- 
tions, according to the criterion 1161 . 



it is a challenging issue to predict the distribution function 
of a galaxy resulting from incomplete violent relaxation. 
Lynden-Bell (1967) first addressed this question by de- 
veloping a statistical theory of violent relaxation. He pre- 
dicted a DF of the form f LB = f LB (e) with f LB (e) < 
which maximizes a certain iJ-function at fixed E, M 
whose form depends on the initial conditions through the 
Casimirs (see Chavanis 2006a). In a dilute approximation 
which seems to apply to elliptical galaxies, the DF pre- 
dicted by Lynden-Bell reduces to the Maxwell-Boltzmann 
distribution / ~ e~@ e which extremizes the Boltzmann 
entropy at fixed mass and energy. However, due to incom- 
plete relaxation, this most mixed state is not reached in 
practice 6 and other stationary solutions of the Vlasov 
equation can be reached instead. For example, ellipticals 
are well relaxed (in the sense of Lynden-Bell) in their inner 
region (leading to an isotropic isothermal core with den- 
sity profile ~ r~ 2 ) while they possess radially anisotropic 
envelopes (with density profile ~ r~ 4 ). One success of the 
Lynden-Bell theory is to explain the isothermal cores of 
ellipticals without recourse to collisions. 

The process of incomplete relaxation, corresponding 
to a lack of ergodicity (partial mixing), could be taken 
into account by developing dynamical models of violent 
relaxation as proposed in Chavanis et al. (1996) and 
Chavanis (1998) to understand what limits relaxation. 
Alternatively, in a series of papers, Bertin, Stiavelli and 
Trenti have proposed other ideas to account for incomplete 
violent relaxation. Stiavelli & Bertin (1987) introduced an 
f{°°> model of the form: 



(oo) 



A(-e) 



3/2, 



-cJ 2 /2 



(e<0) 



(34) 



and / = for e > 0, based on the possibility that the a 
priori probabilities of microstates are not equal due to ki- 
netic constraints. This model reproduces many properties 
of ellipticals but it has the undesired feature of being "too 
isotropic" . Then, they introduced another model based on 
a modification of the Lynden-Bell statistical theory. They 
considered the maximization of the Boltzmann entropy 
(in Lynden-Bell's sense) at fixed mass, energy and a third 
global quantity Q = J J u \e\- 3v / 4 fdrdv which is argued 
to be approximately conserved during violent relaxation 
(Trenti et al. (2005) have checked that Q is reasonably 
well-conserved in numerical experiments of dissipationless 
galaxy formation). This variational principle results in a 
family of f( v > models: 

v/2~ 

ae — d i 



(«0 



A exp 



J 2 

,13/2 



(e < 0) (35) 



and / = for e > 0. These models are able to fit prod- 
ucts of V-body simulations over nine orders of magnitude 

6 This is obvious in the case of self-gravitating systems 
because the isothermal distribution coupled to the Poisson 
equation has infinite mass so it is ill-behaved mathemati- 
cally. However, violent relaxation is in general incomplete 
also in systems for which the isothermal distribution is well- 
behaved mathematically. This is the case, e.g., in 2D turbulence 
(Chavanis & Sommeria 1998a, Brands et al. 1999). 
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in density and to reproduce the de Vaucouleur's i? 1 / 4 law 
(or more general R 1 / 71 laws) of ellipticals (Trenti et al. 
2005). The introduction of additional constraints in the 
variational principle could be a way to take into account 
effects of incomplete violent relaxation. Bertin & Trenti 
(2003) also showed that these models exhibit a sort of 
"gravothermal catastrophe" for collisionless stellar sys- 
tems. They argue that this corresponds to a thermody- 
namical instability where internal collisionality is due not 
to stellar encounters (which are negligible because of the 
very long relaxation time) but to several dynamical causes 
making some relaxation proceed even at current epochs. 
According to the discussion of Sec. 14.11 the maximization 
of S at fixed E, M and Q could also be viewed as a con- 
dition of nonlinear dynamical stability for the anisotropic 
DF (|35|) with respect to the Vlasov-Poisson system. 

6. Conclusion 

In this paper, we have shown that the dynamical (colli- 
sionless evolution) and the thermodynamical (collisional 
evolution) stability of stellar systems can be analyzed by 
similar methods. The thermodynamical stability of stellar 
systems has been investigated using optimization meth- 
ods based on the maximization of the Boltzmann entropy 
at fixed mass and energy (Antonov 1962, Lynden-Bcll 
& Wood 1968, Katz 1978, Padmanabhan 1990, Chavanis 
2002a). In contrast, the dynamical stability of collisionless 
stellar systems described by the Vlasov-Poisson system 
has been customarily analyzed by linearizing the Vlasov 
equation around a steady state and studying the resulting 
eigenvalue equation (Binney & Tremaine 1987). We have 
approached the dynamical stability problem by a different 
method, through the maximization of a Casimir functional 
(or f/-function) at fixed mass and energy. This provides 
a condition of nonlinear dynamical stability. Furthermore, 
this allows us to develop a close formal analogy with the 
thermodynamical problem. Due to this analogy, we can 
use methods similar to those introduced in thermodynam- 
ics (series of equilibria, Legendre transforms, Poincare's 
turning point argument,...) to settle the nonlinear dynam- 
ical stability of a spherical stellar system. Thus, the exis- 
tence of turning points of temperature and energy which 
respectively mark the onset of canonical and microcanon- 
ical instability in thermodynamics also mark the onset of 
instability for barotropic stars and spherical stellar sys- 
tems in the dynamical stability problem with respect to 
the Euler and Vlasov equations. For example, when we 
plot the series of equilibria of polytropes, a turning point 
of "temperature" appears for n > 3 and a turning point of 
energy appears for n > 5; these are precisely the indices 
marking the dynamical instability thresholds of polytropic 
stars and stellar polytropes respectively. These indices had 
been previously found by other authors using very dif- 
ferent arguments (see Binney & Tremaine 1987). In this 
paper, we have shown that the nonlinear Antonov first 
law could be viewed as a manifestation of "ensembles in- 
equivalence" in self-gravitating systems. This is an original 



interpretation of this law. This thermodynamical analogy 
allows us to unify two types of studies (dynamical and 
thermodynamical) that had been previously approached 
by very different methods. 

There are, however, some limitations in our approach: 
(i) Our criteria of nonlinear dynamical stability only apply 
to spherical stellar systems described by a DF of the form 
/ = /(e) with /'(e) < 0. It is not clear how they can be ex- 
tended to more general steady distributions of the Vlasov 
equation satisfying the Jeans theorem. However, in the 
case of anisotropic stellar systems, we have proposed (see 
Chavanis 2003a) considering the maximization of an H- 
function at fixed mass, energy and L2 = J f(r x v) 2 drdv 
which could be viewed as an adiabatic invariant (it is not 
strictly conserved by the Vlasov equation, but it can be 
approximately conserved in some situations) 7 . The ex- 
tremization problem leads to a DF of the form / = /(e a ) 
with e = e + 7^— and /'(e a ) < (r a is the anisotropy radius 
and J — |r x v[the angular momentum). Furthermore, the 
condition of maximum is probably related to a form of 
nonlinear dynamical stability. These intuitive results have 
to be made more rigorous, (ii) To use the turning point 
method, we have been led to confine the system within a 
box. Physically, only the last point in the series of equi- 
libria that corresponds to a self-confined structure unaf- 
fected by the box (the density p(R) = vanishes on the 
boundary) is physically relevant (see, e.g., the complete 
polytrope in Fig. 01 ■ However, the whole curve is needed 
to determine whether this state lies before or after the 
turning point of "temperature" (3 or energy E. Certainly, 
the dynamical stability of self-confined systems (with a 
compact support) could be settled by other means, with- 
out plotting the whole series of equilibria for box-confined 
configurations (see Rein 2005). However, in that case we 
miss the interesting analogy with the thermodynamical 
problem that we have found. Furthermore, there are situ- 
ations where the box is necessary. Technically, the box is 
needed when the system can never be self-confined. This 
is the case for isothermal stellar systems and polytropes 
with index n > 5. One justification of the box in that case 
is to delimitate the region of space where the distribu- 
tion function of the above type holds 8 . For example, the 
box could mimic the effect of a tidal radius in the case of 
globular clusters. On the other hand, we have found that 
the dynamical and thermodynamical methods developed 
in astrophysics can have applications for other systems 
with long-range interactions like two-dimensional vortices 
in hydrodynamics (Chavanis 2002c), bacterial populations 
in biology (Chavanis et al. 2004) and the HMF model 
(Chavanis et al. 2005). For these systems, there are situa- 



7 This idea turns out to be similar to that introduced ear- 
lier by Stiavelli & Bertin (1987) with the conservation of an 
additional quantity Q = J J y |e| — 3l/ / 4 /drdv, see Sec.|S] 

8 Instead of a rigid box, we could consider a system in contact 
with an external medium imposing a constant pressure P. In 
that case, the method of linear series of equilibria still works 
in the so-called isobaric ensemble (see Chavanis 2003a). 
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tions where the introduction of a material box is physically 
relevant. The general study of the stability problems, even 
in idealized situations of box-confined configurations, is 
not just academic but can have useful applications in un- 
expected areas. Therefore, it is important to consider the 
problem in full generality even if we sacrifice for direct 
astrophysical applications. Finally, instead of introducing 
a material box, we can use a confinement in energy space 
by using truncated distribution functions like the Michie- 
King model. In that case the method of linear series of 
equilibria still holds as shown by Katz (1980) who ana- 
lyzed different truncated models and determined the limit 
of stability for each of them. Interestingly, he found that 
the limits of stability are not too sensitive to the form of 
confinement. Note that according to the criterion l|16|l . the 
stability limits found by Katz (1980) can be interpreted 
either as limits of thermodynamical stability or as lim- 
its of nonlinear dynamical stability with respect to the 
Vlasov-Poisson system. 

In conclusion, the method of linear series of equilibria 
is very powerful and allows us to determine the limits of 
stability for various models. In particular, one goal of this 
paper was to show that this method can be used for non- 
linear dynamical stability problems, not just for thermo- 
dynamical stability problems. This offers a connection be- 
tween different types of models: isothermal stellar systems, 
self-gravitating fermions, polytropes, Michie-King models 
(truncated isothermals) etc. Furthermore, they allow us to 
determine their stability limits very easily from a purely 
graphical construction: we just need to plot the series of 
equilibria (3(E) (this only demands that we solve the first 
order variational problem) and then check whether the 
configuration of interest lies before or after the turning 
point of temperature or energy in the series of equilibria. 
In thermodynamics, the first turning point of temperature 
implies a loss of canonical stability and the first turning 
point of energy a loss of microcanonical stability. For the 
nonlinear dynamical stability problem, the first turning 
point of "temperature" (in the sense of our thermody- 
namical analogy) implies a loss of nonlinear dynamical 
stability for a barotropic star with respect to the Euler- 
Poisson system and the first turning point of energy a 
loss of nonlinear dynamical stability for a spherical stellar 
system with respect to the Vlasov-Poisson system. 

Appendix A: The point of marginal stability 

Let us consider a stationary solution of the Euler-Poisson 
system satisfying u = and the condition of hydro- 

static equilibrium JJJJ. The linearized Euler-Poisson equa- 
tions around this solution are 



^ + V.(^u) = 0, 



ddu 
l -di 



(A.l) 



Considering spherically symmetric systems and writing 
the evolution of the perturbation as 5p ~ e at , we get 



r 1 d , 2 r \ 

aop + -z — {r pduj 
r z dr 



0. 



d dS& d& 
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We write 

1 dq 



Sp = 



Aitr 2 dr ' 



(A.4) 



(A.5) 



(A.6) 



(A.7) 



where q(r) = 8M(r) = J Q r 5p(r')4:Trr' 2 dr' denotes the mass 
perturbation within the sphere of radius r. Using the 
Gauss theorem d5&/dr — Gq/r 2 , we see that the Poisson 
equation (|A.6(1 is automatically satisfied. On the other 
hand, the continuity equation (IA.4|) leads to 



Su 



Airpr 2 



(A.8) 



where we have used q(0) = 5u(0) — to eliminate the con- 
stant of integration. Finally, substituting Eqs. <|A.7|) and 
(JA.8(1 in Eq. I|A.5|> and using the Gauss theorem and the 
condition of hydrostatic equilibrium dp/dr = —pd<P/dr we 
finally obtain 



d_f p'(p) d g 

dr \Airpr 2 dr 



Gq 



4irpr 2 



(A.9) 



This is the required equation of pulsations for a 
self-gravitating barotropic gas. It is equivalent to the 
Eddington (1926) equation of pulsations albeit written in 
a different form (see Chavanis & Sire 2005). For isothermal 
and polytropic equations of state, we recover the equations 
studied in Chavanis (2002ab, 2003a). It can be shown that 
the eigenvalues a 2 are real. Therefore, the condition of sta- 
bility is a 2 < corresponding to an oscillatory behavior 
of the perturbation. The condition of marginal stability is 
cr = 0. 

Let us now consider the minimization of the energy 
functional Q . The second order variations can be written 



1 



,-)V = ± l R ^-5<S>dr 
2 Jo dr 



(A.10) 



o 87r pr 2 \dr j 
Integrating by parts and using q(0) — q(R) = 0, we obtain 
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We are led therefore to consider the eigenvalue problem 



d ( p'{p) d 
dr \ Anpr 2 dr 



+ — 



q\(r) = \q\(r). 



(A.13) 



If all the eigenvalues A are negative, then 8 2 W > and 
the configuration is a minimum of W at fixed mass. This 
implies that it is nonlinearly dynamically stable. If at least 
one eigenvalue A is positive, the configuration is a saddle 
point of W and the stability criterion is not satisfied. The 
marginal case is when the largest eignenvalue A is equal 
to zero. In that case, Eqs l|A.9|l and (|A.13I) coincide. This 
implies that the point in the series of equilibria where 
the system becomes linearly unstable (a — 0) coincides 
with the point where it ceases to be a minimum of W, 
i.e. A = 0. Therefore, the onset of linear and nonlinear 
dynamical stability coincides, as previously found in the 
particular cases of isothermal and polytropic distributions 
(Chavanis 2002ab,2003a). 

Appendix B: Passage from F[f] to F[p] 

The optimal distribution function /* which minimizes 
F[f] at fixed density profile p(r) is determined by C"(/«) = 
—(3(v 2 /2 + A(r)). Since C is convex, this relation can be 
inversed to give 

.,2 



/*(r,v) = F 



/?( y + A(r; 



(B.l) 



where F(x) — (C')~ 1 (—x). The density p = J f*dv and 
the pressure p = \ J f*v 2 dv (in d dimensions) can then 
be expressed as 



(3 d /- 



r<7(/3A), 



with 



g(x) =2— S d 



+ 00 



F(x + t) t~~s-dt, 



^ n+OO 

h(x) = -2^S d F(x + t)t*dt, 



(B.2) 



(B.3) 



(B.4) 



where Sd is the surface of a unit sphere in d dimensions. 
Eliminating A between the foregoing expressions, we find 
that p = p(p) where the barotropic equation of state is 
entirely specified by C(f). We can now express F[f] = 
E[f] - TS[f] as a functional of p by writing F[p] = F[f*\. 
The energy is simply given by 

E=^Jpdr+^J pt>dr. (B.5) 

On the other hand, the Casimir I) loll can be written 
2^S d r r+oc 
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pd/2 
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C[F(t + 0\)] t—dt. 
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Integrating by parts and using C'[F{x)] = —x, we find 
that 



S = - 



2 d ' 2 S d 
d(3 d / 2 



,lv I F'(t + p\)(t + f3\)t d ' 2 dt. (B.7) 
o 



Integrating by parts one more time and using Eqs. (|B.2|) 
(|B.3|) and IB .4( 1. we finally obtain 



S = ^1/3 J /.</r + »' / A//./J-. 



Collecting all the previous results, we get 

F[p] = - J p(\ + ^dr+ l -J p^dv. (B.9) 

Finally, using the relation h'(x) = —g(x) obtained from 
Eqs. I|B.3(I and (|B.4|) by a simple integration by parts, it 
is easy to check that Eq. (|B.2|) implies 



A'(P) = - 

so that 

X + P - = - 
P 

Thus, 
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